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1.  Introduction 

“Sensor  coverage  is  the  problem  of  deploying  multiple  sensors  in  an  unknown  en¬ 
vironment  for  the  purpose  of  automatic  surveillance,  cooperative  exploration  or  target 
detection  [5].”  Sensors  can  either  cover  an  area  of  interest  statically  (fixed  sensors)  or 
dynamically  (mobile  sensors).  Static  coverage  can  be  used  when  the  area  or  objects  of 
interest  can  be  completely  covered  by  the  associated  sensors.  However,  dynamic  cover¬ 
age  becomes  necessary  when  the  available  sensors  cannot  adequately  cover  the  area  or 
objects  of  interest  from  fixed  positions. 


2.  Problem  Description 

This  chapter  focuses  on  the  dynamic  sensor  coverage  problem  when  there  is  only 
one  sensor  available  to  cover  N  sites.  This  research  is  an  extension  of  the  work  accom¬ 
plished  by  Tiwari,  et  al.  in  [5].  Each  site  is  defined  to  be  a  discrete  time  linear  system 
located  at  a  unique  point  in  space.  The  sensor  maintains  discrete  time  Kalman  filter  esti¬ 
mates  of  an  attribute  of  interest  at  each  site,  but  is  constrained  to  only  measuring  the  at¬ 
tribute  of  the  site  where  it  is  physically  located  at  a  given  instant  in  time.  An  indicator 
function,  described  by  Sinopoli  in  [4],  ensures  that  if  the  sensor  is  over  site  i  at  time  k, 
both  the  measurement  and  time  updates  of  the  Kalman  filter  are  executed  for  site  i. 
However,  if  the  sensor  is  not  over  site  i  at  time  k,  then  only  the  time  update  is  executed 
for  site  i. 

The  physical  realization  of  this  problem  could  range  from  estimating  the  changing 
temperature  in  several  buildings  to  estimating  the  pollution  emission  levels  of  several 
manufacturing  plants  in  an  area.  In  these  examples,  the  attributes  would  be  temperature 
and  pollution  emissions,  respectively.  The  changing  attributes  of  each  site  are  modeled 
in  the  following  manner.  Consider  N  independently  evolving  linear  time  invariant  (LTI) 
systems,  whose  dynamics  are  given  by 


y,  =Cx,  +v, 


(1) 


where  Xk,  Xk+i,  Wk  &  R  '  andy^  Vk  e  R ’ ,  w  and  v  are  Gaussian  random  vectors  with 
zero  mean  and  covariance  matrices  Q  and  R  respectively.  In  this  chapter,  we  track  only 
one  attribute  per  site.  Since  all  sites’  attributes  are  independent,  the  information  pertain¬ 
ing  to  the  attribute  is  found  on  the  diagonals  of  A,  C,  Q,  and  R.  For  instance,  all  informa¬ 
tion  regarding  the  attribute  at  site  two  would  be  contained  at  A22,  C22,  Q22,  and  i?22- 

The  goal  of  the  Kalman  filter  is  to  keep  a  bounded  estimate  of  the  a  priori  error 
covariance  between  the  random  vectors  and  x^ ;  that  is,  the  true  state  of  the  system  and 

the  estimate  of  the  system  state  at  time  k,  respectively.  The  a  priori  error  covariance,  i.e. 
the  error  covariance  on  the  ‘time  update’  side  of  the  Kalman  filter,  at  time  k  and  location  i 
is  denoted  and  is  defined  as 
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Pr,=E{{x,-xl){x,-x,y]. 


(2) 


In  general,  the  dynamic  coverage  problem  is  said  to  have  been  solved,  if  for  N  sites,  the 
limit  of  the  expected  value  of  the  error  covariance  as  k  approaches  infinity  is  finite  for  all 
sites,  if  Similarly,  if  the  limit  of  the  expected  value  of  the  error  covariance  as  k 

approaches  infinity  is  unbounded  for  some  ^.0  ^  0  ,  then  the  dynamic  coverage  problem  is 

not  solved.  If  the  error  covariance  is  unbounded  at  any  site,  the  system  is  referred  to  as 
unstable;  if  the  system  is  bounded  for  all  sites,  it  is  referred  to  as  stable  [6]. 

The  sensor’s  motion  can  be  modeled  as  an  independently  and  identically  distrib¬ 
uted  (IID)  random  process  or  by  a  discrete  time  discrete  state  (DTDS)  ergodic  Markov 
chain.  This  chapter  only  considers  the  Markov  chain  model.  The  concept  of  steady  state 
ergodic  Markov  chains  is  described  in  depth  in  [2]  and  [3].  For  simplicity,  we  assume 
that  the  sensor  can  move  instantaneously  from  one  site  to  another  at  any  time  k.  The 
DTDS  Markov  chain  has  a  transition  probability  matrix  T,  where  Ty  is  the  probability  that 
the  sensor  will  be  at  site  j  at  time  k+\  given  that  the  sensor  was  at  site  i  at  time  k.  Tu,  the 
/th  diagonal  entry  of  the  transition  probability  matrix,  denotes  the  probability  that  the  sen¬ 
sor  remains  at  site  i  at  time  k+\  given  that  the  sensor  was  at  site  i  at  time  k.  Also,  let  Ki  be 
the  steady  state  probability  of  finding  the  sensor  at  site  i. 

The  conditions  under  which  the  coverage  problem  is  or  is  not  solved  were  already 
discussed  in  a  qualitative  sense;  now  those  conditions  will  be  explicitly  defined  for  sensor 
motion  described  by  an  ergodic  Markov  chain. 

(a)  Let  (A,  C)  be  detectable  and  (A,  -Jq)  be  observable,  then  the  sensor  fails  to  solve  the 
coverage  problem  if  at  least  one  of  the  following  conditions  hold: 


1  ^,(2 

1  -  ;r,  a, 


(3) 


where  a,  is  the  eigenvalue  of  A  associated  with  site  i.  Note:  in  this  chapter,  since  every 
site  is  independent,  all  off-diagonal  elements  will  be  zero,  so  a,  will  simply  be  the  value 

of  ^.[5]. 

(b)  If  matrix  C  is  invertible,  then  the  sensor  solves  the  coverage  problem  if  all  of  the  fol¬ 
lowing  conditions  hold  [5]: 


1  ^,(2  /el, 2,..., A. 

1  -  71,  a, 


(4) 


If  the  coverage  problem  is  solved,  then  a  lower  and  upper  bound  of  the  expected 
value  of  the  error  covariance,  ] ,  can  be  obtained  based  on  tt,,  Tu,  A,  C,  Q,  R,  and 

Pifi.  Reference  [5]  provides  additional  detail  on  these  bounds.  If  the  bounds  diverge  for 
at  least  one  site,  then  the  coverage  problem  is  not  solved.  However,  if  the  bounds  con¬ 
verge  for  all  sites,  then  the  coverage  problem  is  solved.  Equations  (4)  and  (5)  will  later 
be  referred  to  as  feasibility  inequalities.  Figures  1(a)  and  1(b)  illustrate  bounds  for  both 
stable  and  unstable  sites. 
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(a)  (b) 


Fig.  1 .  Figure  (a)  shows  a  typical  unstable  site:  both  bounds  diverge.  Figure  (b)  shows  a  stable  site:  both 
bounds  converge. 


3.  Simulation:  Checking  the  Theory 
3.1  Simulation  Approach 

Intuition  suggests  that  the  faster  the  system  dynamics  of  a  site  relative  to  other 
sites,  the  more  time  the  sensor  must  spend  over  this  site.  Also,  it  makes  sense  that  the 
sensor  must  divide  its  attention  among  the  sites  according  to  the  dynamics  of  the  attribute 
of  each  site.  The  theory  in  [5]  confirms  this  intuition.  We  now  provide  a  simulation  both 
to  demonstrate  the  theory  and  to  provide  additional  insights  into  the  relationship  between 
sensor  motion  and  the  dynamics  of  each  site. 

A  Monte  Carlo  simulation  was  built  in  MATLAB®  using  a  DTDS  Kalman  filter 
with  an  indicator  function  and  a  DTDS  Markov  chain.  The  error  covariance  was  com¬ 
puted  at  each  site  for  all  time  iterations  over  multiple  replications.  The  average  error  co- 
variance  over  the  multiple  replications  was  calculated  for  all  time  iterations  and  plotted 
with  the  appropriate  error  covariance  bounds  for  each  site.  The  values  used  for  equation 
(1)  in  the  simulation  were  C„  =  0.2,  Qu  =10,  and  Ru  =  2  for  all  sites. 

The  feasibility  inequalities  in  equations  (4)  and  (5)  not  only  convey  whether  or 
not  the  coverage  problem  is  solved,  but  also  indicate,  given  the  dynamics  of  each  site, 
whether  or  not  a  given  transition  probability  matrix  provides  a  feasible  solution  to  the 
problem.  Refer  to  Figure  2  for  the  following  discussion.  As  discussed  in  the  Section  2,  if 
at  least  one  site’s  bounds  are  divergent;  i.e.,  if  the  feasibility  inequality  in  equation  (4)  is 
not  satisfied  for  at  least  one  site,  then  the  transition  probability  matrix  does  not  provide  a 
feasible  solution  to  the  coverage  problem.  Note  that  the  bounds  in  Figure  2(c)  are  diver¬ 
gent  and  that  the  inequality  in  Figure  2(f)  is  not  satisfied.  However,  in  Figures  2(d)  and 
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2(e),  the  bounds  are  convergent,  indicating  that  both  inequalities  in  Figures  2(g)  and  2(h) 
are  satisfied.  In  this  case,  the  overall  coverage  problem  is  not  solved  because  the  dynam¬ 
ics  at  site  one  cannot  be  adequately  monitored. 


A  = 


1.3 

0 

0 


0 

1.2 

0 


0 

0 

1.1 


(a)  System  Dynamics 


T  = 


.36111 

.36111 

.36111 


.33333 

.33333 

.33333 


.30556 

.30556 

.30556 


Tt  =  [0.361 11,  0.33333,  0.30556] 

(b)  Sensor  Motion 


(c)  Bounds  for  Site  1 


(d)  Bounds  for  Site  2 


(e)  Bounds  for  Site  3 


1-.36111(2-.36111)  1 

1-.36111 

.63889  >  .59172 

(1)  Inequality  for  Site  1 


1-. 33333  (2 -.33333)  1 

1-. 33333  1.2" 

.66667  <  .69444 

(g)  Inequality  for  Site  2 


1-. 30556  (2 -.30556)  1 

1-.30556  ^1.1" 

.69444  <  .82644 

(h)  Inequality  for  Site  3 


Fig.  2.  This  figure  shows  that  the  bounds  computed  in  simulation  match  the  feasibility  inequalities.  The 
feasibility  inequality  at  site  1  is  not  satisfied,  and  the  bounds  at  site  1  are  divergent.  The  feasibil¬ 
ity  inequalities  at  site  2  and  3  are  satisfied,  indicating  that  the  bounds  at  sites  2  and  3  converge. 


3.2  Simulation  Observations 

3.2.1  Case  I:  System  Dynamics  Equal,  Transition  Probabilities  Varied 

Next,  we  examine  a  case  in  which  the  system  dynamics  are  the  same  across  all 
sites.  The  transition  probability  matrix  is  constructed  so  that  the  coverage  problem  is 
solved,  even  though  each  site  receives  a  different  amount  of  “sensor  attention.”  In  this 
case,  site  one  will  be  visited  by  the  sensor  most  often,  and  site  three  will  be  visited  least 


6 


often  [Figure  3],  Each  dot  in  Fig.  3  represents  the  average  error  covariance  over  10,000 
Monte  Carlo  replications.  There  are  200  dots  at  each  site,  one  for  each  time  step.  As  a 
site  gets  less  sensor  attention,  we  notice  several  effects:  the  bounds  converge  more 
slowly,  the  scale  of  the  bounds  is  increased,  and  more  dots  are  outside  of  the  bounds.  As 
a  site  nears  the  point  of  instability,  the  bounds  begin  to  look  more  like  a  line  than  a  curve. 


'1.2 
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0  ■ 
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(a)  System  Dynamics  (b)  Sensor  Motion 
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g,2000 
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(c)  Site  1 


(d)  Site  2 


(e)  Site  3 


Fig.  3.  This  figure  shows  Monte  Carlo  simulation  trends  when  the  system  dynamics  are  equal  at  each  site 
and  the  sensor  motion  is  varied. 


3.2.2  Case  II:  System  Dynamies  Varied,  Transition  Probabilities  Equal 

In  this  ease,  the  transition  probability  matrix  is  approximately  equal  for  all  sites, 
meaning  that  every  site  will  be  visited  approximately  the  same  number  of  times.  The  sys¬ 
tem  dynamies  vary  aeross  the  three  sites:  site  one  has  the  fastest  dynamies  and  site  three 
has  the  slowest  dynamies  [Figure  4].  The  varied  dynamies  eause  results  similar  to  the 
varied  sensor  attention  in  the  last  seenario.  As  a  site’s  dynamies  get  slower,  the  bounds’ 
seales  deerease,  the  number  of  dots  outside  the  bounds  deerease,  and  the  time  required  for 
the  bounds  to  eonverge  gets  smaller.  Also,  as  ean  be  seen  in  Figure  4(e),  when  a  site’s 
dynamies  are  slow,  and  it  reeeives  suffieient  attention  from  the  sensor,  the  sample  error 
eovarianees  tend  to  form  a  horizontal  line  well  within  the  bounds. 
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Fig.  4.  This  figure  shows  Monte  Carlo  simulation  trends  when  the  system  dynamics  are  varied  at  each  site 
and  the  sensor  motion  is  equal  for  all  sites. 


3.2.3  Summary  of  Observations 

The  Monte  Carlo  simulations  eonfirm  both  theory  and  intuition.  It  is  elear  in  the 
simulation  results  that  as  a  site  gets  more  sensor  attention,  the  average  error  eovariance  is 
more  likely  to  eluster  within  the  bounds.  We  also  note  that  the  slower  the  dynamies  of  a 
site,  the  less  sensor  attention  is  needed  to  ensure  that  the  error  eovarianee  remains 
bounded  and  stable.  Finally,  when  a  slowly  evolving  site  has  suffieient  sensor  attention, 
the  average  error  eovarianee  remains  well  within  the  bounds. 

In  the  next  section,  we  introduce  a  method  that  exploits  the  system  dynamics  to 
find  a  good,  although  not  necessarily  optimal,  sensor  motion  model. 

4.  Optimization:  Finding  a  good  sensor  motion  model  given  the  system  dynamics 

4.1  Optimization  Approach 

In  this  section,  we  present  an  approach  to  determine  a  good  transition  probability 
matrix  given  a  model  of  site  dynamics.  Note  that  the  existence  of  a  method  that  guaran¬ 
tees  an  optimal  solution  remains  an  open  question. 

The  approach  presented  in  this  section  is  a  genetic-based  heuristic  algorithm 
called  scatter  search  implemented  in  MATLAB®.  The  goal  of  this  meta-heuristic  is  to 
provide  both  intensification  and  diversification  of  the  solution:  intensification  meaning 
that  the  algorithm  uses  randomly  created  feasible  solutions  to  progressively  find  better 
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results  with  respect  to  the  objective  function,  and  diversification  meaning  that  the  algo¬ 
rithm  uses  differences  between  randomly  created  feasible  solutions  to  increase  the 
chances  of  finding  the  best  solution  available. 

The  objective  function  is  based  on  the  feasibility  inequalities  discussed  earlier. 

The  objective  is  to  minimize  the  maximum  ratio  over  all  N  sites,  as  shown  in  (5),  subject 
to  typical  probability  constraints  on  n  and  on  the  rows  of  T. 


mint 


max 

ig[1,2,...,V]| 


l-TT, 


a, 


(5) 


This  min/max  objective  will  be  referred  to  as  the  performance  indicator;  the  smaller  the 
performance  indicator,  the  better  the  solution. 

In  general,  the  algorithm  performs  the  following  process.  First,  initial  solutions 
for  T  are  randomly  generated.  A  solution  is  retained  if  it  is  determined  to  be  feasible 
based  on  equation  (4).  Next,  linear  combinations  of  feasible  solutions  are  constructed  to 
form  new  solutions.  Each  solution  is  assigned  a  score  based  upon  the  objective  function 
(5),  and  each  solution  is  ranked  according  to  its  score.  After  a  set  number  of  iterations  in 
which  no  better  score  is  obtained,  the  algorithm  is  terminated.  The  solution  with  the  low¬ 
est  performance  indicator  is  presented  as  the  best  solution  found. 

The  simulation  was  also  used  to  investigate  whether  the  set  of  feasible  solutions  is 
convex.  First,  solutions  were  constructed  so  that  the  solutions  were  feasible,  but  very 
close  to  the  infeasibility  boundary.  Next,  linear  combinations  of  these  close-to-the- 
boundary  solutions  were  formed,  using  equation  (6),  with  X  e  [0,1] .  If  any  linear  combi¬ 
nation  of  two  initial  solutions  was  found  to  be  infeasible,  then  the  solution  set  cannot  be 
convex.  In  fact,  more  than  one  infeasible  solution  was  found,  so  the  solution  set  of  all 
feasible  solutions  is  not  convex  by  demonstration.  The  matrices  shown  in  Figure  5  show 
one  example  where  the  linear  combination  of  two  feasible  solutions  formed  an  infeasible 
solution.  In  this  example,  X  =  0.5,  the  system  dynamics  are  shown  in  Figure  5(a),  and  T\ 
and  T2  represent  the  two  feasible  solutions  in  Figures  5(b)  and  5(c). 


T^=XT,+{\-X)T^ 


(6) 
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Fig.  5. 


4.2  Optimization  Results 


This  section  presents  results  of  the  optimization  algorithm.  The  effectiveness  of 
the  heuristic  is  demonstrated  by  comparing  solutions  generated  by  the  algorithm  with  ran¬ 
domly  generated  feasible  solutions.  Feasible  solutions  are  generated  by  creating  random 
transition  probability  matrices  constrained  so  that  each  row  sum  is  one.  The  equilibrium 
probability  vector,  ku  is  computed  for  each  matrix,  then  the  feasibility  inequalities  are 
checked.  Those  transition  matrices  that  provide  a  feasible  solution  are  saved.  Heuristic 
solutions  are  compared  to  random  feasible  solutions  under  two  cases  that  differ  in  the  un¬ 
derlying  site  dynamics. 


4.2.1  Case  I:  Faster  Dynamics 

For  the  first  case,  one  hundred  feasible  solutions  were  randomly  created  with  site 
dynamics  modeled  by  matrix  A  in  Figure  6(a).  The  mean  of  the  performance  indicators 
from  the  100  random  solutions  was  0.9729,  the  sample  standard  deviation  was  0.0246, 
and  the  minimum  and  maximum  values  were  0.9010  and  .9997,  respectively.  The  heuris¬ 
tic  solution  had  a  performance  indicator  of  0.8901,  a  9%  improvement.  Figures  6  and  7 
show  a  graphical  representation  of  a  randomly  created  solution  and  a  heuristic  solution, 
respectively.  The  main  difference  between  the  figures  is  the  scale  of  the  bounds.  Also, 
there  are  more  dots  outside  the  bounds  in  Figure  6  than  in  Figure  7. 


Avergage  Error  Covariance  ^  Avergage  Error  Covariance 
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;.  6.  This  figure  shows  the  graphical  depiction  of  a  representative  randomly  created  solution  with  a  per¬ 
formance  indicator  of  0.9737. 
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Fig.  7.  This  figure  shows  the  heuristic  solution  in  graphical  form  for  the  case  of  faster  dynamics.  The  per¬ 
formance  indicator  in  this  case  is  0.89007. 
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4.2.2  Case  11:  Slower  Dynamics 

In  the  second  comparison,  one  hundred  feasible  solutions  were  randomly  created 
with  slower  dynamics,  modeled  by  Figure  8(a).  The  mean  of  the  performance  indicators 
from  the  100  random  solutions  was  0.9398,  the  sample  standard  deviation  was  0.0469, 
and  the  minimum  and  maximum  values  were  0.7624  and  0.9991,  respectively.  The  heu¬ 
ristic  solution  had  a  performance  indicator  of  0.7027,  a  25.2%  improvement.  From  the 
two  comparison  scenarios  representing  faster  and  slower  dynamics,  we  note  that  the  heu¬ 
ristic’s  performance  is  better  when  the  dynamics  are  slower.  Figures  8  and  9  depict  a 
randomly  created  solution  and  a  heuristic  solution,  respectively,  for  the  case  of  slower 
dynamics.  In  both  figures,  the  error  covariances  tend  to  form  a  straight  line  within  the 
bounds.  The  main  difference  between  the  figures  is  in  the  mean  of  the  average  error  co- 
variances.  Also,  the  magnitude  of  the  bounds  in  Figures  9(c),  9(d),  and  9(e)  is  consis¬ 
tently  less  than  in  Figures  8(c),  8(d),  and  8(e). 

One  observation  from  Figure  9(b)  is  counter-intuitive.  We  would  expect  a  site 
with  faster  dynamics  to  warrant  more  sensor  attention  than  sites  with  slower  dynamics, 
but  the  heuristic  solution  in  this  case  does  not  bear  this  out.  We  note  that  ks  is  greater 
than  III  and  K2,  even  though  site  three  has  the  slowest  dynamics  [Figure  9(b)].  Intuition 
would  suggest  that  ns  should  be  the  smallest  steady- state  probability. 
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Fig.  8.  This  figure  shows  the  graphical  depiction  of  a  representative  randomly  created  solution  that  has  a 
performance  indicator  of  0.941 1. 
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Fig.  9.  This  figure  shows  the  heuristic  solution  in  graphical  form  for  the  case  of  slower  dynamics.  The 
performance  indicator  was  0.7027. 


5.  Summary  and  Future  Work 

The  goal  of  this  chapter  was  to  demonstrate  the  theory  of  the  dynamic  sensor  cov¬ 
erage  problem  in  the  case  of  a  single  sensor  whose  motion  is  modeled  by  a  DTDS 
Markov  chain.  The  Monte  Carlo  simulation  demonstrated  that  the  theory  developed  in 
[5]  matched  both  intuition  and  empirical  results.  Another  goal  was  to  show  that  an  im¬ 
proved  transition  probability  matrix  can  be  found  using  a  heuristic.  An  improvement 
over  random  feasible  solutions  was  found  using  scatter  search,  though  this  method  does 
not  guarantee  a  global  optimum.  The  search  did  provide  solutions  that  were  sometimes 
counter-intuitive.  The  heuristic  performed  best  in  cases  with  relatively  slower  system 
dynamics.  The  algorithm  run  time  depends  on  the  exact  conditions  specified,  but  for  the 
conditions  specified  in  this  research,  the  optimization  algorithm  required  from  two  to  four 
minutes  of  run  time  on  a  desktop  PC.  We  also  showed  by  demonstration  that  the  set  of 
feasible  solutions  is  not  convex. 

Future  work  will  include  refined  optimization  approaches  for  the  single  sensor 
case  and  extensions  to  multiple  sensors.  For  the  single  sensor,  an  effort  will  be  made  to 
understand  why  the  heuristic  produces  results  that  are  counter-intuitive  with  respect  to  the 
division  of  sensor  attention,  and  to  further  explore  under  what  conditions  the  scatter 
search  heuristic  performs  best.  The  heuristic  will  also  be  used  to  provide  insights  that 
might  lead  to  a  theoretical  solution  to  the  problem  of  determining  an  optimal  transition 
matrix. 
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